Adaptive cubic overestimation methods for unconstrained optimization. Part II: worst-case iteration complexity

An Adaptive Cubic Overestimation (ACO) framework for unconstrained optimization was proposed and analysed in Cartis, Gould & Toint (Part I, 2007). In this companion paper, we further the analysis by providing worst-case global iteration complexity bounds for ACO and a second-order variant to achieve approximate first-order, and for the latter even second-order, criticality of the iterates. In particular, the second-order ACO algorithm requires at mostO(ǫ) iterations to drive the objective’s gradient below the desired accuracy ǫ, and O(ǫ), to reach approximate nonnegative curvature in a subspace. The orders of these bounds match those proved by Nesterov & Polyak (Math. Programming 108(1), 2006, pp 177-205) for their Algorithm 3.3 which minimizes the cubic model globally on each iteration. Our approach is more general, and relevant to practical (large-scale) calculations, as ACO allows the cubic model to be solved only approximately and may employ approximate Hessians.